Htd: Heterogeneous task decoupling for two-stage object detection
Abstract
Decoupling the sibling head has recently shown great potential in relieving the inherent task-misalignment problem in two-stage object detectors. However, existing works design similar structures for the classification and regression, ignoring task-specific characteristics and feature demands. Besides, the shared knowledge that may benefit the two branches is neglected, leading to potential excessive decoupling and semantic inconsistency.
Methodology
To address these two issues, we propose Heterogeneous Task Decoupling (HTD) framework for object detection, which utilizes a Progressive Graph (PGraph) module and a Border-aware Adaptation (BA) module for task-decoupling.
1. Semantic Feature Aggregation (SFA) Module: We first devise this module to aggregate global semantics with image-level supervision, serving as the shared knowledge for the task-decoupled framework.
2. Progressive Graph (PGraph) Module: Performs progressive graph reasoning, including local spatial aggregation and global semantic interaction, to enhance semantic representations of region proposals for classification.
3. Border-aware Adaptation (BA) Module: Integrates multi-level features adaptively, focusing on the low-level border activation to obtain representations with spatial and border perception for regression.
4. Instance-level Semantic Consistency (ISC): Finally, we utilize the aggregated knowledge from SFA to keep the instance-level semantic consistency of decoupled frameworks.
Experimental Results
Extensive experiments demonstrate that HTD outperforms existing detection works by a large margin, and achieves:
• Single-model 50.4% AP on COCO test-dev set using ResNet-101-DCN backbone
• 33.2% APs (small objects) on COCO test-dev set
• This is the best entry among state-of-the-arts under the same configuration
The code is publicly available at https://github.com/CityU-AIM-Group/HTD.